1. Technical Filed
The present invention relates to an image decoding apparatus, an integrated circuit, an image decoding method, and an image decoding system each of which is intended to sequentially decode macroblocks included in a coded picture, in a predetermined order.
2. Background Art
In recent years, contents, such as moving images, still images, audio, and text have been generally treated in a uniform manner with the developments in multimedia applications. The uniform treatment of the contents is possible by digitalizing all the contents. However, since the digitalized images have enormous amounts of data, the compression techniques for information of the moving images are essential for accumulating and transmitting the images.
On the other hand, in order to mutually use the compressed video data, the importance lies in the standardization of the compression techniques. The standards of the compression techniques for the moving images are hereinafter referred to as video compression standards. The examples of the video compression standards include H.261 and H.263 ratified by the International Telecommunication Union Telecommunication Standardization Bureau (ITU-T). In addition, the video compression standards further include Moving Picture Experts Group (MPEG)-1, MPEG-2, and MPEG-4 of the International Organization for Standardization/International Electrotechnical Commission (ISO/IEC). The video compression standards also include the H.264/AVC (MPEG-4-AVC) developed by the Joint Video Team (JVT) of both the ITU-T and MPEG.
Under the video compression standards, one picture is divided into blocks each having a predetermined number of pixels (16 pixels by 16 pixels of luminance components), and coding or decoding processes are performed per block. The block to be a processing unit will be referred to as a macroblock.
Furthermore, under the video compression standard, one macroblock are divided into sub-blocks each having a predetermined number of pixels (8 pixels by 8 pixels of luminance components), so that coding and decoding methods are switchable per sub-block. The sub-block to be the processing unit is referred to as a sub-macroblock. The macroblocks and the sub-macroblocks will be hereinafter collectively referred to as macroblocks.
Generally, one of the important processes for the video compression coding is motion compensation inter prediction coding for compressing information amounts by reducing the redundancy, in a temporal direction, of consecutive pictures (images) composing moving pictures. The picture to be referred to in coding a picture to be coded will be hereinafter referred to as a reference picture.
The motion compensation inter prediction is a method of detecting a direction in a reference picture and a degree of a motion for a macroblock in the picture to be coded, generating a predictive image, and coding a difference between the generated predictive image and the picture to be coded. The reference picture is a picture prior to or subsequent to the picture to be coded in display order.
The information indicating the direction in a reference picture and the degree of a motion for a macroblock in the picture to be coded is referred to as a motion vector.
A picture in which intra prediction coding is performed without reference to any reference picture is referred to as an I-picture. Furthermore, a picture in which motion compensation inter prediction coding is performed with reference to only one reference picture is referred to as a P-picture. In addition, a picture in which motion compensation inter prediction coding is performed with reference to two reference pictures at the same time is referred to as a B-picture.
The reference picture can be specified for each macroblock. The reference picture prior to the picture to be coded in a coded bitstream is a first reference picture in display order. In addition, the reference picture subsequent to the picture to be coded in a coded bitstream is a second reference picture in display order.
When a B-picture is coded, there are cases where macroblocks included in the B-picture are coded in a coding mode called a “direct mode”. The direct mode is a coding method (coding mode) for calculating a motion vector for each macroblock using a motion vector of another macroblock previously coded, without coding a motion vector.
A picture to be referred to for calculating a motion vector for the macroblock coded in the direct mode will be referred to as a co-located picture. Furthermore, a macroblock in the same spatial position as that of a macroblock to be coded in the co-located picture is referred to as a co-located macroblock.
More specifically, when a macroblock is coded in the direct mode, the motion vector for the macroblock is calculated using a motion vector of the co-located macroblock.
The conventional processes for decoding a macroblock coded in the direct mode will be hereinafter described with reference to FIG. 14.
First, as a precondition for decoding a macroblock coded in the direct mode in the future, predetermined amounts of motion vectors for decoded macroblocks in a picture are stored when the macroblocks are subsequently decoded in accordance with the H.264/AVC.
For example, motion vectors for the maximum 32 pictures are stored in accordance with one of the specifications of the H.264/AVC. Assuming that 1 picture includes 8160 macroblocks, motion vectors for 261120 macroblocks in total need to be stored.
A normal buffer is not sufficient for storing such an enormous amount of motion vectors in an image decoding apparatus, which is unrealistic. Thus, the motion vectors each corresponding to a decoded macroblock are generally stored in a memory, such as a Dynamic Random Access Memory (DRAM), outside the image decoding apparatus that decodes images.
The macroblock is sometimes abbreviated as “MB” below. Furthermore, the motion vector is sometimes abbreviated as “mv” below.
As illustrated in FIG. 14, when a macroblock of a B-picture is decoded in an MB decoding process, a macroblock type indicated in a header of the macroblock is referred to, and it is determined whether or not the macroblock has been coded in the direct mode (S3000).
When the macroblock has been coded in the direct mode, a Direct Memory Access (DMA) instruction for obtaining, from an external memory, a motion vector for another macroblock to be referred to is transmitted to a DMA controller that is not illustrated (S3001). Then, with a motion vector calculating process at Step S3002, the motion vector for the macroblock is calculated using the motion vector transferred from the external memory.
Meanwhile, with the conventional technique illustrated in FIG. 14, after the determination that the macroblock has been coded in the direct mode (S3000), the external memory is accessed (S3001) and the necessary motion vector for another macroblock is obtained.
In other words, since transferring the motion vectors from the external memory to the image decoding apparatus is necessary, there is a problem that the wait time occurs as illustrated by an arrow in FIG. 14 and a necessary time T for decoding the macroblocks of the B-picture is prolonged.
As a technique for solving the problem, for example, PTL 1 discloses an image decoding apparatus. The conventional image decoding apparatus will be described with reference to FIG. 15.
A co-located macroblock corresponding to a macroblock in an s-th macroblock is hereinafter referred to as a co-located macroblock for an s-th macroblock, where “s” is an integer equal to or larger than 1. The co-located macroblock for the s-th macroblock is a macroblock corresponding to a motion vector to be used for calculating the motion vector for the s-th macroblock.
For example, when “s” is equal to (n+1), a co-located macroblock corresponding to an (n+1)-th macroblock is hereinafter referred to as a co-located macroblock for an (n+1)-th macroblock.
In FIG. 15, a “MB (n) header process” indicates a process in which a header of an n-th macroblock is referred to. Furthermore, a “MB (n) decoding process” indicates a process in which an n-th macroblock is decoded. Furthermore, “Motion vector calculation (n)” indicates a process in which a motion vector for the n-th macroblock is calculated.
Furthermore, “Transfer instruction #(n+1) transmission” indicates a process in which a transfer instruction for transferring a motion vector of a co-located macroblock for the (n+1)-th macroblock is transmitted to a buffer. Furthermore, “Mv#(n+1) transfer” indicates a process in which the motion vector of the co-located macroblock for the (n+1)-th macroblock is transferred to a buffer and stored in the buffer.
Furthermore, “Case where mvCol is unnecessary” in FIG. 15 indicates a case where a macroblock subsequent to a macroblock to be processed is not coded in the direct mode. Furthermore, “Case where mvCol is necessary” indicates a case where the macroblock subsequent to the macroblock to be processed has been coded in the direct mode. Furthermore, times T1 and T2 are time periods necessary for decoding corresponding macroblocks.
The image decoding apparatus in PTL 1 executes a process of decoding the n-th macroblock (S4000) in parallel with a process of transferring the motion vector of the co-located macroblock for the (n+1)-th macroblock to a buffer (S4001) as illustrated in FIG. 15.
Thus, when the (n+1)-th macroblock has been coded in the direct mode, in the case where the (n+1)-th macroblock is decoded during the time (period) T2, the motion vector can be immediately calculated using the co-located macroblock for the (n+1)-th macroblock stored in the buffer (S4002).
As a result, the time T2 necessary for decoding the (n+1)-th macroblock is reduced. The parallel process is performed in the same manner on each of the n-th, (n+1)-th, (n+2)-th, . . . , (n+x)-th macroblocks. The (n+x)-th macroblock is the last macroblock included in a stream. Thus, the same advantage can be obtained from any of the macroblocks.
As such, even when an access latency to the external memory is large, the effect on arithmetic performance can be eliminated by obtaining motion vectors in advance.